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. Abstract . • 
• ft. 

This paper describes ^e ^development and pilot test results of a 
computer-based curricular package? dealing *rtth instruction in word prob- 
lem solving for elementary school students* An iniori^tion processing 
model for solving word problems is used to sequence the problems. Pilot 
test results with five students suggest that the sequence derived from the 
model is hierarchical, that students learn from the program, and that 
they enjoy ^the experience* ... ^ 
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COMPUTER -ASSISTED INSTR UCTION IN WORD PROBLE.MS; 
^ A ' > DESIGN AND ASSESSMENT ' 



DESIGN AND ASSESSMENT 
tlicholas C. Laudato and Richard A. Roman 



/ ^ University of Pittsburgh 



This paper describes the* develophient and pilot test of a currjiculatr 
package dealing with instruction in word problem solving for the elemen- 
tary school. I'he design and rationale for the curricular package are 
detailed in another paper (Roman & Laudato, 1974). The curriculum, 
designed the Word Problem Program, %is one of several programs dc- 
*v^Joped to investigate uses of thfe computer for instruction in problem 
'solving. It* was created as part of the Comput'er-Assisted Instruction in 
Problem Solvjlng Project, v/ith the fundamental goal of teaching el^men- 
^ry school children some^eneralizable probicm solving skills. This 
groj.ect is* in turn, conducted as part of the Oakleaf Computer Resource* 
Project initiate by the Learning Resear6h>jid Development Center in 
1969 under sponsorship of the; Nationaf Science .Foundation. The princi- 
pal purpose of the Oakleaf Computer Resource I^roject is'to "irfvestigate 
appropriate and effective uses of the Computer in an individualized school 
xn order tc^foster the adoption of individualized systems of elementary 
educa'tion" (Blocfc, Carlson, Fitzhugh, Hsu, Jacobson, Puente * Rom^il, 
Rosncgr,. Simon, GlaseY^ Cooley, 1973, ^. 1). , ^ 

* * , ' Ci- 

The Word Problem Program is designed to accomjmsh -tyro instruc- " 

tional goals, to teach^ elementary school c^^hildren how to solve arithrrietic 

word problems, arfd to tea oh problem solving skills applicable to situations 

not involving word problems. All instruction in the Word Problem Pi»o- 
* *^ 

gram is accomplished by means of a computer program* 



All the topics presented in this paper are discussed in more detail* 
Jtr\9L doctoral dissertation by Laudato (1975). Because the scope of this 
study was so vast, the writejrs were forced to limit the nui^ber of tojSiq^. 
and their' depth' of discussion in its presentation* The topics presented 
were selected in an atte;inpt to both summarize the study and highlight its 
significance. This paper describes the |^oals of the program, a method- 
ology for defining and seque!;icing instructional objectives based on an 
informatiqa processing model, and the design feature S| that enable the pro- 
gram to meet these goals. ^ Finally, it summarizes the results of an experi- 

mental -pilot .te st. *" 

♦ 

Goals ot the Word Problem Program r 

A word problem is a written representation of a concrete, physical 
situation that requires the problem solver to engage in purely mental 
activities in order to reach a quantitative solution. In the^ord Problem 
Program, the problem solving task is viewed as requiring the application^ 
of two distinct sets of skills for solvtion: word problem- specific ^skills 
and general problem solving skills* To specify the word problem- specific 
^ skills, we have designed a model of human behavior on word problem 
solving tasks based on the information processing work of Bobrow (1968) 
and Paige and Simdn (1966). According to the model, people so)Ve word 
problems by: 

(a) translating individual D>{rases into symbolic e^pressi^ons. 

(b) combining the exp^^sions to form equations. ^ 

(c) manipulating (if necessary) the equations to attaiaa directly, 
solvable equation. « « ^ 

(d) performing arithmetic computations to attain,the answer* 

. While the language used to describe the model is algebraic, elemen- 
tary school students do not use formal algebra to solve v/oi>d problems^ 
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♦ • \ , . V » 

The algebraic l^inguage, then, is a mfetaphbr for the processes of sym- 
bolic representationTarMf manipulation lhat actvtally do'take place* 

^ . ' '■" * - ' 

^ Tne .information processing model clarifies the skills required to 

, s61ve word problems* ""The skills of the third and fourth steps are usually 
algorifhnvic to elementary school students, while the skills of the first 
two steps include higher Its-vel cognitive processing* 'in the Word Problem 
^Program, concern is with problem solving and not with comipetence in 
algorithmic manipulation, and thus students are assumed capable of^olv»* 
*mg number sentences and computing accurately* The first major goal-^of^'^ 
the Word Problem Program is to provide an environment that facilitates " 
the acquisition and application of skills necessary to perform the first 
^ and second steps of^the mpdel* 

Several researchers (Newell & Simon, 1972; Oltdn, 196'9; Papert, 

-l972i^Polya, 1957, Wickelgren, 1974) have hypothesized skilly and st rate* 

gies for solving w^ord problems whose application is not limited to the 

specific case of aritkmetic word problems, bu^t are gener$ilizable to many 

> 

situations. ^iTiese general pi:oblem solving skills include recognizing the 
existence of a problem' solving situation and clparly stating a goal; gather- 
ing, r-ecording, and organizing the data, breaking a complex problem into 
parts, solving the P^^ts, an^ reintegrating the partizU solutions into a 
solution to the whole problem, and recognizing when^^b^eolution has been 
attained, when help isr needed, or when to give up. The second major 
goal of the Word Problem Program is tp provide an environment that 
facilitates the acquisition and application of these skills, 

, I nstructional Objectives ' ^ 

An instru^onal objectfve for the Word Problem Program consists 
of a group o£ problems that require the same kinds of ii^form^ation process 
ing skills. Thus, in order to sp^cifly an instructional obj|^tive, we nfeed ^ 
to sjJecify the skills that the student must be able to apply and the criteria 
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for evaluating that application. Instructional objectives a»tY sequenced so 
4 that later objectives require quantitativ^ely more ajid/or qualitatively dif- 
terent processing skills. . The specification of skills foV^ach objective 
a^d, consequently, the hierarchical or^fiering of objectives; is facilitated' , 
by a component analysis performed with respect to the word problem- ^ . 
specific skills identified by the model, "jo^^complish the component 
analysis, we have"9cvised an algorithm which solves the problems us,ed 
'in the Word Problem Program and outputs data about the j>t ocessing steps 
required to r'each a solution^ " These processes ^re analogous to the skills 
humans employ, to solve word problems* 

The solution algorithrtr^ is a set of rules that translates a given word ^ 

problem into symbolic £o,rrr> and manipulates the symbols to attain a solu- 

tion to the probleiri. The algorithm operates on item forms.^ An item 

form specifies a set of 4>roblems whici/have identical syntactic form 

(Hively, 1963; Osburn, 1968)# In this case, each item form specifies a ^ 

set of problems created by filling several grammatical slot.s with items 

from word lisfk. For example, given the following word list and sample 

. - . - ^-.< ^ - ^ ^ . _ L- 

item form, 108 (4x3x3x3) unique word probletr^s can be generated 

without varying the numerical data. ' . 

Wo^d Lists: * i> , , 

I 

(NAME) = JOHN, SUE, JOAN, DICK . . - 

(VERB) = COLLECTED, OBTAINED, GATHERED 

(OBJECT) = STAMPS, APPLES, BALLS 

(DATE) =• AUGUST, THE SPRING,, MAY ^ 

(PKONOUNy = HE, SHE - 

V • -_Samp\e item^ Eorj3i: ^ . . ^ 

DURIIJG (DATE), (NAME) (VERB) (NUMBER, 1) (OBJECT), - 
SINCE THEN, (PRONOUN) (yERB)^ (NUMBER, Z).MORE (OBJECT). 
. ' HOW MANY (OBJECT) DOES (PRONOUN) HAVE NOW? 
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Exampla Problems: " "y* ^ ^ 

' / . ^ r 

l5URING*THE SI^^ING, JOHN COLLECTED 25 STAMPS. 
SINCE THEN, HE COLLECTED 13 MORE STAMPS. ^ 

^ HOW mAny stamps d^'es he have now? 



DURING MAY, SUE OBTAINED 16 APPLES. 
SINCEf THEN, fcHE (RETAINED 32 MORE AIJPLES. 
HOW MANY A>RLES DOES SflE HA VE NOW? 



In the Word Problem Program, there are 45 differ enyword lists, 



each containing from 3 to 10 words. Thus, the number of uniq^je ^yo^d 

problems that can be generated from each item form reaches the hundred 

thousand^. ( A typical it^m form that uses's^jc listp with approximately 

\f 

sjBven wd/d6*per lisft c<^n generate 7^= 1 17,649 ditferent problems. ) In 

' , y i. ^ ^ % " « 

addition, the specific numbers used in each problem yary to further 

diversify the problems. A more extended discussion of the use of item^ 

forma is (oxfnfiin Romiri a'nd Laudato (1974), • , 

A 'set o'f approximately 200 item forms define* the universe of prob- 
lems for use in^e,\Vord)probiem-Prog5*am, ! "Hiese problems require 
f^fcom one to threi^different arithmetic operations applied from one to five 
times. Th6y are roughly cqui valient in difficulty^ to those from thp Indi- 
vidualize^d Mathematics and Ixidividually Prescribed Instructionvmatc rials 
for grades 3 to 7 (Research for Better Schools U learning Research and 
Development CSiter, 1972). 

The solution ^gorithmis applied in two stages. #a translation stage 
and a maifipulation stage. The translation stagd makes a symbolic repre- 
Bentation of each phrase in the Item form and passes 'the resulting, equa- 
tions to the ipanlpulation stage. Ihe manipulation stage conr^binc9 the equa^ 
tions in such a ^y as to produce a solution. The follo^ng sections illus- 
trate the operation of each\f these stages. For simplicity^ the examples 
will show the algc/ritJim opera ting^n problems instead of Item forms, - 
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The Tranalation Stage • ^ 

The first task that the solution algorithm xnust perform in the transr 
lation stage is that of dividing the problem into phrases* Under specified ^ 
conditions, cev»in words and/or punctuation serve as ghrase delimltexs* 
e.g. , periods, \}iie St ion marks, or the word "IF** followed by text that Is, 
in turn, followed by "AND" or "rfOW MANY. " After phrases have beeti 
identified, the algorithm proceeds to translate each phrase into a symbolic 
representation. Each phrase is interpreted as cither assigning a numeti* 
cal value to a variable or asking .a question. Hie. algorithm ignores phrases 
that do neither of these. Thus, a ph^jase such as "JI^ AND TOM WALKED 
TO THE ZOO" would not be represented symbolically in the resultant set' 
of equations. * * 

\ * 
Some word problems*^ require temporal information in order to inter- 
pret and solvfi* them. The algorithm notcl the occurrence of phrases such 
as "TWO WEEKS AGO," "YESTERDAY," "NOW/' and so on, and tags 
each expression as to the relative temporal sequence of the events in the 
problem. If there ^re fto sucK temporal references in (he problem^ this 
informi.tion is omitted, j 

Moot phrases assign a piece of n\imerical data to some object, 
person(s). or event? in the problem situatibn. For example, the phrase 

V ) * 

(1) JIM BOUGHT 15 APPLES 

assigns to a variable named **the number of apples of Jim" the value "15." 
This assignment is represented symbolically as 

(2) {APPJl-ES, JIM) = 15 • * \. • ^ 
Similarly, the phrase 

(3) TOM BOUGHT 5 MORE APPLES THAN JIM 

asslg^ns to a variable named "the number of more apples than Jim of Tom" 

• ' . • 

6 
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the value "5. " This is ^represented as ^ 

(♦) (MORE APPLES THAN JTIM, TOM) = 5 

£toth of th^se translations follow the general x^e which states that If 

* • J \ ^ f 

phrase consists of a string words (represented as STRING, 1> ) fol- ^ 

V ' , " ' 

lowed by a ver& which connotes positive, acquisition (represented as 

^+VEHB>) followed by a number (rt;presented as < NUMBiRrl>)ioUowfed 

by a second string of words (<S^RING,2>), or, ^ * 

(5) <STRENG. 1> <+VERB> <NUMBER. 1> <STRING..2> ^ ' 
thczi^such an expression is symbolically represented as 

(6) (<STRING.2>, <STRING.,1>) = <NUM]^ER. 1> 

. Thus, if the algor^tiim finds a phrase which matches the form of 
(5), it translates it to (6}* A phrase such as | 

1(7) JHOW MANY APPLES DID JIM AND TOM BUY? 

must be viewed as 

^ (8)^' W AND TbM BOUGHT ? APPLES 

to*be translated, by^the same general rule, to 

' (9) (APPLES. JIM AND TOM) = ? 

Some symbolic expressions in the form of (6) must be further trans « 

lated« For example, expression (4) above, 

• } 
(MppE APP^JES THAN JIM, TOM) = 5 

is transformed by a specific rule to 

(10) (APPLES. TOM) = (APPLES. JIM) + 5 

That is, the statement *'the number of more apples than Jim of Tom is 5" 
is equivalent to the statement "the. number of apples of Tom la equal to 
tiie number of apples of Jim plus 5. " . 



Finally, the algorjthip scans thfc set of generst^*^ expressions for 
further •modification by ineans of specific rules such as iHe union rule. 

* This rule states tliat ' 

(11) (X.,Y, AND Z] = (X, Y) + (X, Z), where X, Y and Z ar< toxt. ^ 

Application of this rule to expression (^^isnc rates th<£sadditiunaLl%l(prcs 

* sion 

(IE) (APPLES. JIM AND TOM) = (APPLES, JIM) ^ (APPLES, TOM). 

The solution algorithm for the translation stage 6onsii}ts of a act of 
rules which perform such tran8f9rinations on each phrase ufxvcry item 
form in the domain of the program. This stage is compfete when ea.ch 
resultant ^xprpssion has ^een reduced to its simplest fotm. The sym 
bolic expressions arc then passed to the manipulation stage* 

The Manipulation Stage ^ 

The naanipulation stage of the algorithm consists of a set of rulf s 
governing the coxi.«inatiun anu manipulation of the expressions dctivcd hy 
the translation stage. Hxc first step m the manipulation stage is to locate 
the expression which contains the unknown* If this expression i* in Indl 
rcct.form, it must be manipulated so that £hc question ma/K will be iso - 
lated on one side of the equaUon. An analogy to number sentences helps 
illustr?.te the concept of direct versus indirect equations. Thc^numbcfr 
sentence = 25 + 19*' is in direct form, no processing other than arith* 
metic is nccessaryUn ordc^r to solve for The ifumber sentence 

"> - 5 ^ 12'*, however, is in indirect form, in otder to attain the solution, 
it must first be transformed to the equivalent number sentence *"* 12 i 5* 
and then the value of can be determined by arithmetic* 

Several rules gove^rn tVic manipulation ol equatibns in order to attain 
desirable results. For exam'ple, » » 



(W) ''(X, Y) YA = (X;.Z) if and only if (X, Y) = (X, Z) - A 

(14) ^(X,^ Y) • A = pC, Z) if and only if (x', Y) = (X, Z)/A, 

, ^ . where ^, Y, Z are text and "A" is.either a nximber or "? 

• After the unknown has-been isolated, the algorithm checks wheth<^r 

the equations are tagged according to temporal sequence. If this is so, an 

external equation is ad^^ed to tife previously generated list. This equation 

simply states that an fevent occurring in the past is modified by an evpnt 

. o<#curring over time to produce the current event, or 
•* , * 

(15) (X, Y) [PAST] + (X, Y) [OVERTIME] =:''(X, Y) [PRESENT]* 
^ t 

* The algorithm now proceeds to make direct substitutions of expr^es- 
sions ^or those occurring in the equation that contains the unknown, ^ it 
locates, in the unknown equation, an expression for whicjti no^^^ssignment 
Vis directly av^ai^ble, it searches for the presence of that expression in 
o|her equations. Jhe algorithm must then proceed to manipulate the new 
equation to isolate the desired term. If this process does not succeed, 
other external equations ir^ay be ad'ded to the list of equations. For exam- 
ple, a multiplication equation which is oft^n employed is 

(16) (XKY) . (Y, Z)^p?, Z) ^ 

This can be interpreted as follows; Th^ phrases , « 

(17) JOHN HAD 20,BOXES OF ORANGES. 

(18) EACH BOX CONTAINED 10 ORANGES. * 

(19) - HOW MANY ORANGES DID JOHN HAVE? 

are translated to \ 

(20) (BOXES, JOHN) := ^0 . ^ " 

> ► * 

(21) (ORANGES, BOJ^) =10 

(22) (ORANGES, JOHN) '= ? ' * 

Applying equation (16) to equations (20), (21), artU (22) yields 

(23) (ORANGES, BOX) • (BOXES, IJOHN) = (ORANGES, JOHN) . 



Tv/o direct 5xibstltytic^s can now be performed on actuation (Z3) to yield 
(24) . 10 • 20 = {ORANGES, JOHN) = ? , 

In applying the algorithm, each step is recorded to yield a "trace*' 
of the process. Let us take as an example the phrases translated in the 
preceding subsection* The phrases were: . - * 

(1) JIM BOUGHT 15 APPLE'S. 

(3) TOM BOUGHT 5 MORE APPLES^ THAN JIM. 
(7) HOW MANY AT>PLES DID JIM AND TOM BUY? 

* • 

« 

These s^re easily translated to • ^ ^ 

(2) (APPLES, JIM) = 15 ' ' 

(4) (MORE APPLES THAN JIM, TOM) = 5 

(9) (APPLES, J^M AND TOM) = ? * . 

Before leaving the translation stage, equation (4) is further processed to 
yield ' 

(10) (APPLES, TOM) = (APPLES, JIM) + 5 . 

and equation (9) generates an additional equation by applicatipn of the union 
rule (11). 

"(1^ (APPLES, JIM AND TOM) = (APPLES, JIM) + (Af>PJ1.6s, TOM) 

In the manipulation stage, equation (9) is identified as the unknown. 
Since it is direct and does not involve sequence, a substitution is made of 
equation (12) into (9) to yield ^ ^- 

(25) ? = (APPLES, JIM) + (APPLES, TOM) 

Next, two direct substitutions of (2) and (10) into (25) yield 

(26) ? = 15 + (APPLES, JIM) + 5 
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and finally; (E) is again substituted into (26). to give 
(27) ? '1= 15 + 15 + 5 = 35 

J 

This problem thus required, in ^addition to the translation skills, the 
applicatign of the union rtile, th^ special »'more than*' rule, and four directs 
substitutions in order to attain the solution. Jhe algorithm has been applied 
to each of the 200 item forms to produce lists of processing steps similar to 
that in the example above. The remainder of this section shows how these 
data were used to define and structure learning objectives, 

» ' t 

The Objective Array 

The solution algorithm outputs lists of the steps needed to solve a^ 
glve'n\tem form. According to the model of h^man problem solving behaVr 
ior, these steps aire analogous to the information processing steps that 
humans utilize in solving word problems. In this se^se, the application of 
the algorithm to each iten> fqrm is equivalent to performing a component 
analysis. What was needed was a way to use inforrnation from the solution 
algorithm to create and sequence instructional objectives. 

The list of steps from^the solutjLon algorithm was examined in^detail 
to identify similarities and differences among item forms. It was imme- 
diately ascertained that some item forms weue nearly identical in terirjB 
of the processing steps needed for solution. Such item forms should not 
differ significantly from e^ch othfer in terms of.either difficulty or pre- 
requisite skills. It was thus decided that the item forms should be'divided 
into a shia^ll number of groups that were, in theory, homogeneous with 
regard to information processing skills. 

Several important distinctions between item forms were evident. 
First, the item forms differed on the basis of the number of direct sub- 
stitutions required to solve them. This dumber varied from one to five. ^ ^ 
Item forms that require more direct substitutions require a greater amount 

il 
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of j>r.oce8aing and thus, hypothetically, are more difficult. Next, approxi- 
mately^ half of the item forms required the use of rules to inanipulate equa-^ 
tions from indirect to direct form. Item forms that require this additional 
information processing step are, hypothetically, rpore difficult than those 
that-4o not. Another important distinction is that several of the problems 
contain numerical data, unnecessary for the solution of the problem* Ke- 
search has demonstrated (Hydle Clapp, 1927) that problems containing 
superfluous data ape more difficult th<\n problems that do not. Finally, 
item forms can be dj-stinguished on the basis of whether or not they require 
additional external rule? for solution. * . 

With these distinctions serving as guidelines, the item forms^ were 
divided into 2f4 groups which were then placed in a three dimensional array. 
This 4x3x2 objective array is illustrated in Figure 1. Item forms in the 
top layer of tlie array (in cel|s 13-24) are those that require the application 
of one of the *'indirect rules" while those in the bottom layer do not* Approxi 
mately the same number of ilfem forms are in each layer. The three rows 
of the objective array represent the dimension that involves the number - 
of -direct- substitution data. Item forms in the first row require one^r 
two^direct substitutions, i^ost in, the second row require three, and thode 
in the third row require four or five substitutions. These groupings were 
chosen so thai almost the same number of item forms woul4 lie in each row* 

TJhe last dimension of the objective array makes further distinctions 

*. ' 

between these six (2 layers by 3 rows) categories. Item forms in the first 
column require only direct substitutions in.o^rder to reach the solution. 
Those in the second column require the use of special multiplicative^ rules 
in addition to direct substitution. Item forms in the tF*ird column require 
the same processes as the second but also require the use of special exter- 
^ nal rules such as the temporal sequence equation. Finally, those item 
forms m the fourth column -require the rules of the second column but also 
contain superfluous numerical data. The objective array, thus structujfed, 

/ 

\ » ' ' 
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Indirect rules reqaired:^ * 



1 -2 substitution steps 



3 substitution steps 
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Direct rules only; 



4*5 substitution steps 



Direct Direct External Superfluous 
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only and explicit required contained 

' .Kiles » " 

^ f 



Figure 1. Array of instructional objectives for item form types. 
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groups all of the item forms in the program and organizes them according 
to prerequisite relationships* ^ * 

o * 

Along each dimension of the .array, the item forms increase in diffi- 
culty. and build upon the prepfequisite item forms that preceded. Foi exam- 
ple, in order to solve the item forms in the third column successfully, the 
titudent must possess the skills required to sofve item forms in the first two 
columns. This is true for both layers and for every row. Further, to solve 
item fornjs in the second column, the student, must possess the ^skills reqdired 
to solve itein forms'in ^e first column. Similarly, this type of prerequisite* 
relationship holds true for each, dimension of thenar ray. The e:£cept^on is ^^^^ 
the fourth column. Only the first and second columns, but not the third cot- 
umn, are prerequisite to the fourth column. ^ ^ 

. \ ^ ' 

Since the item forms within each group are nearly homogeneous in 

terms of the skills neede.d for solution, the groups o( the at ray can serve 
as instructional objectives. For example, group 13 defines an objective as 
such: The student will be able to solve word problems that require one or 
two direct substitutions and application of an indirect rjile. Graphically, 
the immedia.te prerequisites for a given objective are; (a J the objective in , 
the layer b^elow it, if any, (b) the objective to the left of it in the row, if any; 
and (c) the objective above it in tjic column, if any. A given objective also 
has as^rerequisites all of the prerequisites of its prerequisites. For 
exSmple, the hnmediate prerequisites of objec'tive 18 are objectives 6, 17, 
and 14. Ih addition, objective 14 has as pre requi site % 2 and 13; 17 has S and 
13; 6 has 5 and 2, and 2 has 1 as a prerequisite. Thus, the prerequisites^of 
.obj[ective 18 are objectives 1, 2, 5, 6, 13,' 14, and 17. The objective array 
can therefore 4«pict a quite complex learning hierarchy in a relatively sim- 
ple fashion. , / j * 

In practite, the instructional objectives of the Word Probt^m Program 
actually require the s|;udent to be able to solve a mix of problems from sev- 
eral groups. TTie rationale for this decision) which essentially seeks to 
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insure that students will not solve problems with trivial algorithms, will be 
discussed further in the nexi section* The method employed to accomplish 

this mix of proBlemd required tlie designation of a ' vtarget" group and one or 

\ 

more "practice" groups. In the instructional session during the pilot test, 
the student received a set of problems randomly selected from the designated 
groups. Fifty to sixty five percent of the problems were selected from the 
target group- - the group for which the student's work is evaluated > -and the. ^ 
remainder were selected from tue practice group. For each group in the 
top layei of the array, the practice group was the group beneath th^ target 
group, fo^ each group in the first rov/, the practice group was the group >0 
the left, and, for each of the remaining groups, the practice group was \]^e ^ 
group immediately above it in the same column. In each case,, the^.prac- ^ 
tice group is one of the previously mastered imxnedi%.te prerequisites. 

\ 

, Students are evaluated on the basis of their performance on. the prbb- 
lems from the target group. A student's work on a target, group is desig- ^ 
nated as either mastery or nonnmstery, and sequencing decisions are^^de \ 

\ 

on the basis of thir evaluation. 

The objectiv^ array is used to individualize the sequence of instruc- 
tional objectives for'^each student. This sequencing is based on both the « 
student's performance and on the hypothesized prerequisite relationships 
among the groups. After having mastered group 1, the student moves along, 
a chosen dimension of the array until either the. last group in that dimension 
is mastered or the student cannot master a given group. In either case, a 
new direction is chosen and instruction begins in like manner aloi^g t^^at 
dimension. If progress has been stopped along all dimensions by non^nas- 
tery, irts tru^tion' begins. \yith the first nonmastered group as the target 
group. The program will fir^^t attempt, to proceed across rows, then down 
columns, and finally, up to the, top layer. This precedence is subject to 
change, however, depending on whiih direction is easiest for the student. 
For example, suppose a student mastered objectives 1 and 2 but failed on 
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objective 3, The program would then choose a new direction, and thus 
group 5 wouldVe tKe^nc/kt target group. If the .stux^ntj^stered objective 5* 
and proceeded to master objective 9, a r^w direction ^puld again be chosen. 
It the student then failed objective 13, the nonma9*ered group 3 would be pre- 
jTented again. Faih)re here would again allow llie student to proceed a^ong 
/the rows to objectives^ 6 and 10. Thus, although th6 student must eve^ktually 
pass, objective 3, initial or eVen subsequent failure on it will not impede 
progress in o&er'^reas of the arr.ay* ' , 

^In summary, Jhe ablution algorithm and objective array specify instruc- 
tional objective?, structure them in terms of theiif relationships, and are 
used to make sequencing decisions. « # , 

Instructional Strategie's 

't 0 

/ All instruction in the Word Problem Program takes place by means 
of the computer. The program presents prOblems^to the^ individual student 
at a time- sharing terminal, interacts witii'the student to asdist m the solu- 
tion pro^cess, jt^jlges the student's work, and chooses new problems appro* 
priate to the student's state of learning. Students ate expected to ^.cquire 
the target problem solving skills by induction. Although specific iostruc- 
.tional strategies are employed to assist in this acquisition, the sequence of 
instructional objectives bjsars the brunt of the instructional load. The Word 
Problem Program combines the traditional instructional methods tl'iat have 
shown the greatest success. This section outlines several instructional 
features of the progi^m. ♦ ^ 

The program emphasizes translating the problem to number sentences* 
An analytic instructional method would encourage the student to proceed 
systematically with the translation. Several methods, which are all called 
analytic, have been studied. For example, Morton (19^5) defines l^^i^'analytic 
method as a three step process, whereas. Washburne and Osborne, (1926) 
use a siic< step process. All definitions, however, include three; crucial 
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steps: (a) determine what is to be found, (b) determine what xe^tionships 

•» • 

^Ipnd numerical data are given; and (c) decide which mathematical operations 
are needed to attain the solution. v 

In their informatiye review, Suydam and Riedesel (V^^) conclude that 
*'itiformal procedures are superior to following rigid steps , . , if the analy> 
sis method is used, it is recommended that^only one or two of the steps be 
tried with any one problem*' (p. 50). Rather than rely exclusively on analy- 
sis, a second method, relying on analogous problems, has beea incorporated 
in the Word Problem Program. 

The analogy method has a variety of definitions in the literature. All 
include two crucial parts, (a) a problem similar to the original is presented 
by the instructor, and (b) all the informati^Jn necessary for the correct solu- 
tion is preserved in tlie analogous problem (Gorman, 1967, Morton, 1925). 

In thisjyord Problem Program, the analysis method and the analogy 

method are^ used when the student a^sks for a hint. The set of hints for each 

problem models the translation process. * 

• « » . * * - 

The first hint for each problem identities the unknown for the student 
and suggests that he or shp reread the problem. This hint G,or responds to 
the first step of the analytic method. The seconds hint is a resta^iement of 
the problem in simpler syntactic form, omjitting suj^erfluous information, 
«The student is directed to compaiv^ tho restatement to the original. This 
^second hint corresponds to the analogy method. The third hint is a mathe- 
matical statement relating the variables in the problem «vnd^giving an appro^ 
pridte number sentence. This hint corresponds to the second step of the 
analytic method by making the relationships between variables explicit. 
Since we assume students know ho^ to solv^ number sentences, no hiift 
corresponds to the third step of the analytic method. An example clarifies 
the relationship between hints and the original problem. 

17 



The Problem: 

IfO GROIJPS OF WOMEN AND 19 GROUPS OF PUPILS WENT TO 
THE BAI:l park. HOW MANY MORE PUPILS THAN V/OMEN WERE 
f ^'^ERE AT THE BALL, PARK IF THERE WERE^9 PEOPLE IN EACH 
GR*OUP? . • ' • ' ^ . 

First Hint: " . f ^ 

HINT: RESTATE THE QUESTION. '*J'IND THE NUMBER OF MORE * 
, * PUPILS THAN WOMEN AT THE BALL PARK. 
NOW REREAD THE PROBLEM. ^ 

. ^ \ ; ^ 

Second Hint: 

. HINT: RESTATE THE PROBLEM. IT IS SmiIAR TO:. . 

10 GROUPS ^F 9 WOMEN AND 19 GRdUPS OF 9 PUPILS WEI^E 

AT THE BALL PARK. ^" . 

? = I&ORE PUPILS THAN W^OMEN. ] 

NOW REREAD THE ORIGINAL PROBLEM. 

* 

Itiird Hint: ' 

/> 

HINT: TRANSLATE THE PROBLEM TO A NUMBER SENTENCE: 
10x9 WOMEN + ? WOMEN =19x9 PUPILS, OR . 
10 X 9 + ? = 19 X 9 ^ 
NOW, SOLVE THE^JVMBER SENTENCE.- 

* 

The hi^t structure used in the program is designed to encourage the j 
student to take the following steps in problem analysis.^ (a) identify the 
unknown, (b) translate the word problem into a simpler analogous problem, 
(c) translate the simpler problem into an arithmetic number sentence, and 
^ (d) solve the number sentence. This structure combines the best features 
of the analytic and analogy methods, providng help when it is neeaed and 
modeling desfirable strategies for solving word problems. 
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One final contingency must be provided for in order to ensure the sue 
cess of the program. This contingency deals with the subtly distinctions 
between problem solving, and algorithmic behavior. 

. , ■ . ■ • / 

In most elemehtaty school curricula, word problems are used as 
exercises to increase <;pmpetence in computation. For example, a lesson 
designed to teach the student to multiply three-digit numbers will often be 
followed by a set of word problems that requires the multiplication of j 
three digit numbers, the student can. solve the problem without reading A. 
^ single word.^ In their report on research and develcrpment in elementary ^ 
\ school mathematics, Suydam and Hiedesel (1969^ state that studies- "reveal 
that pupils often give little attention to the actual problems, instead they 
almost randomly manipulate numbers" (p. 47). R. P. Stevenson ^cited in 
Riedescl, 1967) pointed out that manipulation of the numbers is more algo- 
rithmic than random. For example, he describes a method used by an ele- 
mentary school student: , 

If there are lots of numbers, X add. If there are only two nxmibers 
with lots of parts, I subtract. But if there are just two numbers 
with one littler than 8ie other, it is hard. I divide If they come 
out even, but if they don't, I multiply. (p./308) 

In extreme cases, the numbers alone cue the correct opif^ation, and 
the student responds algorithmically rather than by utilizing problem solving 
skills. The order in which problems, are presented can also radically alter 
the difficulty of individual problems. For example, both LoftSs (1970) and 

4 t 

Hydlft and Clapp (1927) found problems are easier when they can be solved 
by the same operations (in the same order) as the pxp^ding problem. Stu- 
dents of len respond to a new problem by attempting to apply the solution 

steps from the previous problem. 

e • - • 

To avoid reinforcing such undesirable algorithms, the Word Problem 
Program employs three strategies. First, the problems presented by the 
program are written so that no word or phrase consistently cues the cor- 
rect pperatvbn. For example, the words "divided by, " "times as many as, 
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*'each, and "average*' are used in both multiplication ^n^ division problems.^ 
Second, problems arc sequenced so that students never sec a set of problems' 
tliat can all be solved with the same operation. This dis.courages responding 
with the operation used in the preceding problem, finally, the numbers in . 
the Word Problem Program are chosen so that they vary in magnitude so 
tha^ne of the numbers is a flic tor of another^ r<igardfess of ,\vhich arithme- 
tic operation correctly solves the problem. This fea^re eliminates the cuc& 
that the numbers themselves provide and, along with the first two feah^res, 
forces the student to find more appropriate strategies for choosing a particu^ 
lar operation. * , * * * 

A goal of the Word Problem Program is to t each the transl^^on of a 

word problem to a number sentence, but not to teach the*^olution of the 

equations ^ the computation of th^ final answer. For this reason,, as well 

as to allow the freedom to choose numbers by the above criteria, the'pro- 

gram must do the arithmetic .^a^cuJat^on for the student. The Word Pi^oblem 

Program requires that the student specify the operations and operandi^ neccs> 
, *■ * * » . . 

^sary to solve the problem, but does not assume that they have the ability to 
compute with the large numbers used in the problems* ^ 

The Pilot Version 

^ $ 

The Wo/d Probjem Program operated on a tryout basis at the small 
computer resource at an elementary school in suburban Pi^^sburgh. The 
computer (a DEC PDP 15 with 64K 18 -bit words of memory) has been ^ 
located in a van outside the school since Spring L972, and now provides 
locally controlled service tb the school throughout the day. 'in^pitc of the . 
coxjtputer's sm&ll siae, it has a general- purpose tirpe - sharing system 
(J^SS) designed and implemented by Learning Research and Develop- ^ 
>ment Center (Fltzhugh, 1970). ETSS currently supports 16 terminals. ^ 

A student's interaction with the Word Problem Program begins with 
a problem written on tlie screen of a cathode -ray tube, ^The numeric data 
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presented In the problem is then listed beneath the problem (in order.of 
appearance in the problem) and labeled with consecutive alphabetic chara'b- . 
ters, each on a separate line* After reading the problem, the student has 
several options available: He or she may type '^CHANGE" to get a new . 
problem, type ''HINT'* to receive a hint, type "ANSWER" followed by a let- 
ter to indicate that t^e number represented by the letter is his/her answer, 
or perform an arithmetic operation. To perform an operation, the student 
types' tWo operands (represented by alphabetic symbols) separated by an 
<ger%tion*symbol {i, ^*{^^ I)* computer then performs the calcula- 

tion and displays the result labeled with the next available alphabetic charac- 
ter. E^ich^time the (Computer processes a request, it erases the request 

and any superfluoy^s displays from the screen before it creates the, new 

^\ L. ''^ ' ' * 

display, . ' i ♦ 

^ If the student types an incorrectly formatted request, the program 
diagnoses the mistake and provides corrective feedback in the form of mes- 
aages. For example, if the student typed "1^ + 24" the program would 
remifid him that he should ''TYPE LETTERS ONLY". If the student desig- 
nates an in«>rrec^ Value for the answer, the program types "WRON,j!; AN- 
SWER'^ followed by i^he promgl "TYPE 'HINT' IF YOU WANT SOME HELP", 
if the answer is correct, a bell is sounded and the message "GREAT!" is 
typed on the screen, ^ 

Pilot Teat ^ ^ ^ ' \ 
^ , A, 

Throughout the developmental period, the Word Problem Program has 
undergone formative evaluation. The program^ has been reviewed by other 
members of the Coniputtjr -Assisted Instruction in Problem Solving Project 
fin each of its*design and experimental pliases. It has also been tested on 
four stuc/ents in a^ experimental non-comp^tei version and with four students 
in an experimental compvter version that requ^* ed ^intervention by the experi- 
menter. Both of these tryouts were conducted in the LRDC laboratories* 
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The results of^thcsc evaluations helped refine the program and encuaj;ag<^d 
continued developmental effocts/ This section reports the results of a third 
pilot tesi^ which was conducted in a school setting with miniaial intervention 
by thtf experimenter* ^ - . 

The pilo^ test was performed with two fourth -grade and three fifth- 
gra^e students at an elementary school, t'our of the students were chusen 
by their teackers for participation ihe pilot test on the basis of their need 
for remeiiaL instruction in word problems. The fifth student was chosei\ 
because his n^acher felt he lacked motivation, required special attention, 
and was disruptive in the normal individualized classroom environment. 

Each student received instruction from the program for two sessions 
per week over a seven-week period. * Hie sessions lasted from 10 to 40 
minutes, with a norm of approximately 25 minutes. Ihe first session ton- 
'siated of an introduction to the format"^ of the program and a 10-dtem pre- 
test wliich sampled ten cells from the a4;ray. In the last^session, the pre- 
test was repeated using thfk same item fprms^ * » 

The* sessions were held during the time set aside each day for the 
students to engage in activities of their choice, provided they have fulfilled 
m^r weekly commitments Students left their rooms at the specified time 
, to come to the centrally located terminal area for instruction. Students 
had their sessions individually with the experimenter sitting^ beside thorn 
^af the terminal, , ' , ' 

The experimenter ass^sed the student's performance and chose 
^ the next instructional objective, *slnce these features had not yet been pro- 
grammed. Interactions with the students were limited to casual conversa- 
tion, instructions as to which objective to ask for (this occurred once or 
twice a session), and, occasion9.11y, questions iconcerning the stu(lcnt*s 
rationale for any particularly interesting behavior. Anecdotal records of 
the student's solicitejd b,nd unsolicited verbalizations were taken by the 
experimenter each time they occurred. ITiese records supplemented the 
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Ite history of every uat^e^action between the student and the program, 
whi,cli was taken by £he 'program. ' , 

In this stage of forma'^ve evaluation, we sought ansVers to the fol- 
lowing questions: ' ' # ^ • ^ 

^ (a) Could shidents learn to respond quickly and, efficiently in the 
format required by the prograixi? - 

(b) Could students^ solve a greater variety;^ of problems after the 
instructignal period tl^ji before? ^ ^ 

(c) Did the hypothesized hierarchical relationship hold true for 
the students studied? 

fd) Did students enjoy and value the experience of working on 
I the program? _ ^ 

^Obviouily» Uiis list does not exhaust the questions wo inteI^d to 
answer concerning the program. Because of the time and programming 
Iimitatipns, however, we were forced to delay the evaluation of the extent 
to which the program accomplishes its problem solving goal«, the effec- 
tiveness of each component instructional strategy and program feature, 
the empirical validity of the array, and many more* Hie remainder of 
this section presents data that provides tentative answers to the four ques*^ 
tions stated above. 

Students ^.ere able to master a se^ of number sentences and simple 
introductory problems in 10 to 15 minutes during the first session* These 
problems were "heavily supplemented by instruction from the experimenter* 
After this introduction, no additional instruction was given in the response 
format. The students made few errors in Inputting commands during sub- 
sequent sessions* ITius, students ^ere able to learn the response format 
quickly, easily, and effectively. The ease with which students mastered 
the command format contrasts with the earlier experience of l^oftus (1'970) 
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with a computer-based word problem test* In her work, students took 
four to eight weeks to master the response format. The Loftus work is 
described in the design document for the Word Problem Program (Roman 
& Latiilato, 1974), and the changes we nnade to improve the response for- 

To answer the second and third questions posed above, we can make 
use of the pre- and posttest data collected by the program. *Figure Z 
graphically depicts the ten cells of the array which were sampled for the 
tests.^ 
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Figur* 2. Cells of the array sampled in the pretest. 



The results of the pretest,, instructional sessions, and posttest are 

r 

summarized in Figure 3 for all five students. Each rectangle represents 
a layer of the array, the direct layer is pictured below the indirect layer. 
A solid black box within a rectangle represents mastery of tb problem or 
objectivci an "X'* represents nonmastery, and a blank means that the 
problem or objective was not tried. This latter case occurred only once. 
Student One's performance on ilic first six problems m the pretest v/as so 
poor that, to avoid alienating him, the session was terminated. 
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Figure 3, Tryout data. 
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Hie set of rectangles labeled ''Instruction** depicts the status of the 
atray for each student at the end of the instructional period. An "X" in a 
box of one of these arrays Indicates that the student worked one or more 
times on^that objective but failed to meet the mastery criterion. It should 
also be noted that a single solid black box may represent several instruc* 
tional sessions during which the student^ attempted to master the objective* 
The graphs depict a positive change in the~-&tu<^ents* problem solving behav- * 
ior as measured along the dimenfsions the array. While such a sxtiall 
samplo cannot definitively prove the program works^ these results are 
encouraging. • * * , - 

In all cases but that of Student Two, the posttest performance was 
superior to the pretest performance. Student Tliree made an error on a 
problem (from cell 1) on ^e posttest that she answered correctly on the 
pretest. This is even more ^usual given that, from her instrug^ional 
array, she had mastered objectives which have, as prerequisites, all of ^ 
the skills needed to solve that particular problem. Of the II problems 
from cell 2 that Student Three saw during instruction, 10 were performed 
errorles^y. Hie student wlearly should have been able to solve the prob^ 
lem on the posttest. Hiere were three other disparities between thf post- 
test data and the data from the instructional sessions*. In these cases^ the 
student demonstrated minster y of, a particular cell, but answered the post- 
test problem from that cell incorrectly. 

A dramatic increase in performz^nce was demonstrated by Student 
One, a fifth-grade student who experienced difficulty on the pretest and 
in the first few instructional sets. By the end of the instructional period, 
the student's perfornnance was equal to the entry level of the other students. 

A rearrangement of the pre- and posttest data will help in answering 
the third question concerning the hierarchical^ relationships between cells 
of the arj^^y. Figure 4 depicts the results on both tests for all students 
in ten 2x5 rectangles- -one for each problem from the test. Each 
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rectangle also contains two columns. The first represents the pretest score 
.and the second the posttest score* As above, a solid black box represents 
a correct answer on the problem, an "X'* represents an incorrect answer, 
and a blank indic^ ♦ss that the student did not ^receive that problem on the 
test. 

The rectangles (problems) are connected by lines to illustrate the 
relationships between problems. One can consider each problem as repre- 
senting the instructional objective as s^ich. The student should be able to 

solve problems requiring the information processing skills stipulated by 
f 

the problem's position in the array. The figure is constructed using the 
hierarchical relationships between objectives in the array. Thus, prob- 
lems that lie below a given probleip are prerequisite to Xt* They are not 
always the only prerequisites, however, and thus the figure does not repre- 
sent a conventional learning hierarchy. 

•I 

For the given set of data, the hypothesized prerequisite relation* 
ships are validated if the following condition holds true. If a particular 
problem has been mastered, then all prpblems beneath it in ihe figure 
should have been mastered, or equivalently, if a given problem was not 
mastered, then no problems above It in the figure should have been mas* 
tcred. The rectangles contain data for ten different cases, that is, a pre* 
'^and a posttest for each of the five students. Thus, Figure 4 represents 
ten different WPst^ of the validity of the relationships between problems. 
Of the ten cases, nine perfectly meet the condition stipulated above. The 
exception is Student Three's posttest performance on problem 2, which* 
was discussed previously. Even if this exception could not be rationalized, 
the results would still be convincingly positive. 

In order to answer the fourth question, data was collected informally 
throughout, the tryout pe >>d. The students' attitudes and motivations were 
noted during each session, and each comment the students made with 
respect to the program was^recorded. The attitudinal and behavioral 
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data will be reported first, followed by discussion of the comments of each 
indi vidua 1* s tudent* 

. Of the 62 8 essions for the five students, there were seven (11%) 

during which the student behavior fell to simply being willing to receive; 
during 45 sessions (73%), the students were willing to respondi and in the ' 
* remaining ten sessions the students exhibited satisfaction in response* 

The next paragraphs make the behaviors exhibited more precise. 

In sessions that wl^e judged*as ^U^g to receive, the students were' 
either inattentive, not task oriented, or behaved indifferently towards the 
program. When called upon to come to the session, they came reluctantly 
or slowly* ^ q 

* In sessions judged as willing to respond, >^the students came early for 

their work, waited impatiently for the previous student to finish, or left 

their 'previous task quicl^ly. Students attended to the display carefully. 

Conversation dealing with topics other than the program was minimal and 
* • 

usually limited to the beginning and end of the ses sion* The students gen- 

/ ♦J 

erally smiled and appeared relaxed and happy with the v;^ork^ 

In the sessions judged as satisfaction in responding, the students 
were even m6re positive. They hurriedjy and **xcitedly came to the ses- 
sion. They smiled, laughed, and joked a go6d deal during the session, 
but were attentive to the task. They often read the problems aloud with 
enthusiasm and laughed at thq occasional rhyming problems or those with 
many or large numbers. Hiey were obviously enjoying themselves greatly* 
Most pf-otested when told the session was ov:ir. , 

The anecdotal data. was also largely positive, Ihis data will be 
exajnined here student by student. Student One'^ reactions were rqixed. 
During the introductory set of problems he exclaimed, "This is neat!*' 
On the sec^t^id flay, he came running out of his classroom to the terminal 
center and shouted, "I've been waiting for^this day!" • 
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^Between the second and third sessloni Student One worked on the 
program by himself, apparently in his free time. This behavior was dis- 
couraged, but indicates his interest and enthusiasm, pn later occasions, 
he complained about both indirect problems and multi-step problems, "I 
don't like the ones like that" and '1 don't like these problems. " During 
the ninth session, he refused to work on any multi>step problems- -"just 
for today." He also complained that his work on the Word Problem Pro- 
gram interfered with bis required school work- -"I'm not getting my work 

r 

done**- -and even missed one sessi^on,in <in attempt to nrialce up classroom 
work, after a one-week absence due to illness. However, he also cOm- 
plained in session three. when the experimenter missed a day, and in i^ea- 
sion^fqur, he .pleaded for an extra session to malce up for the one the experi- 
menter missed. He suggested we Wv^ the make-up session at lunch time 
or after school. Twice he indicated that he preferred to work on the intro- 
ductory set and the number sentence problems, "They^'-e O. ft. Hiked the 
other ones better" and " Now can we do number sentences?" Finally, in 
one instance, he refused to terminate the session by typing *'STOP*'. 
Instead, he quickly pressed RETURN to get a new problem before the 
experimenter could intervene. Although Student One was the only stifdent , 
to offer negative comments, he made the greatest gain in posttest pe^* 
formance. ' 

^Student Two's comments cannot be taken literally. Ihe smile on 
her face and the laugh in her voice indicate her true feelings more than 
the words she used. In one instance, she laughingly stated, "This com- 
puter's wacky" and repeated her message to several passing students. 
She indicated her pleasure in solving problems with large numbers on 
several occasions and twice demanded, "I*Want big numbers. " She also 
jokingly responded to a series of several problems which contained many 
numbers with, "Ooh, I hate these kind. " , * 
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Student Three, although she read virtually every problem aloud 
(perfectly and rapidly), did not offer many spontaneous comment^. One 
anecdote reveals her strong feelings towards the program. When the 
experimenter, arrived at her homeroom for the fourth session, he found 
a party in progress* Class had been cancelled and the students were play- 
ing gam^s and eating candy. When she saw ihe experimenter, she iirnne* 
diately prepared to leave for the session. Her teache'r interrupted and told 
her that she did not have to go on the computer today, the experimenter 

agreed, indicating that they could make it up another day. She insisted, 

ft * 

hoArever, that she wanted to have the. session and would return to the party 

Student Four seemed to enjoy the program more than anyone else. 
On several occasions, she.vtrbally indicated her enjoyment of both rhyme 
problems and multi-a^ep problems. On. two occasions, she smiled and 
commented- -*'0h, lhate these kind*'- -in reference to more difficult prob- . 
lems. ,Once she objected when told the session was over and prolonged it 
an extra 15 minutes to almost 40 minutes- -the longest session in the ^experi- 
ment. * 

Student Five also was positive about the program. Like Student One, 

he insisted that we make up for the lost day at lunch time. One conversa- 

tion in particular, however, illustrates his feelings toward the program. 

Bet;ween problems in the fourth session, he asked the experimenter, ''Did 

you like school when you were a little kid? " The experimenter responded, 

"Sometimes, *' and Student Five exclaimed, "Me, never - - but I like this . I 

J- i 
realhj^Hl^e this!" 

In summary, all students seemed to respond to the program at a level 
which 13 at least equivalent to Krathwohl's "Acquiesence in Responding" 
(Krathwohl, Bloom II. Masia, 1964). The student is active in responding 
but passive in the initiations of behavior. There are m^ny instances of 
"Willingness to Respond": The student chooses to engage in activities 
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within.the program and in proble^p solving. There*are somcL instances of 

"Satisfaction in Response". The student demonstrates "a feeling of satis- 
faction, an emotional response, generally of pleasure, zest, or enjoyment" 
(Krathwohl et ah , 1964, p. 130). 

^ Conclusions 

The results of the pilpt test provided useful feedback for the redesign 
and refinement of several program components. Posiflve results were 
attained f^;: each of the four questions as^ed at this stage of formative 
evaluation. A more detailed study is currently underway which seeks to 
answer other Questions in a more rigorous fashion* This study involves 
28 students wording t>n a vctsion of the program requiring no intervention 
•;by.the experimenters. ^ , ' 

, ^ By far, the most intriguing product of our work on the program is 
the aJlgorithm and array* The solution algorithm represents a methodology 
for performing a rigorous component analysis oi Complex cognitive tasks* 
This allows the precise specification of learning objectives and the pr,e> 
requisite relations between them* The objectiv^,array represents a method- 
ology for organizing the data from a component^ analysis and for making 
sequencing decisions which facilitate a high degree of individualization* 
We belie\*e that these methodologies are gen^raUzable to other s^ituations 
involving complex cognitive tasks* We are begmning to investigate such 
an application to the design of a curriculum to teach competency in solving 
series. This investigation and other studies should provide data to assess 
the true significance of the methodologies* 
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